Technical Field
Color image processing and display, and in particular, rendering of color images that are perceived to be of high aesthetic quality.
Description of Related Art
Current techniques for the rendering of high quality color images are mostly “frame” based. Other techniques for rendering of high quality color images are “region” based. One issue with region based techniques is that they are computationally complex. Additionally, they have been most effectively used for processing of individual color images, rather than a sequence of images that are provided at a high rate, such as a sequence of images in a movie or video. To a large extent, this is because in region-based image processing and rendering, the size of regions needs to be large, many artifacts are produced, and changes in contrast are not rendered well. Consequently, the images rendered and displayed from a video or movie sequence are often not perceived by viewers to be of adequate quality. At the very least, it can be said that there is considerable opportunity to improve the image quality, such that when a typical observer (such as a customer in a theatre) is presented with a video or movie of improved image quality, the will appreciate the improvement over prior viewing experiences, i.e., the observer “will know it when he sees it.”
One known method that is used in frame based image processing and display is Contrast Limited Adaptive Histogram Equalization (CLAHE). CLAHE is typically applied on a frame-by-frame basis. FIG. 1 is a flowchart that depicts the steps of a basic algorithm of CLAHE. The steps performed in executing the algorithm are as follows:
1) In step 10, a movie frame or image is “read,” i.e., provided as a digital image or sequence of digital images. The digital images are comprised of image pixels that are typically defined using RGB tristimulus values.
2) In step 20, for image pixels defined by RGB tristimulus values, optionally, convert the RGB tristimulus values to values of another color space, such as YUV, HSV, or IPT.
3) In step 40, for the entire image frame, compute the histogram h(n] of the luma or intensity pixel values. (For an eight bit grey scale, the range of the values will be from 0-255.
4) In step 45, apply a predetermined/programmable threshold to the image; and in step 50, uniformly redistribute all the histogram values that exceed the threshold among the remainder of the histogram bins.
5) If any histogram values of the revised distribution exceed the threshold, then repeat step 50, redistributing all the histogram values that exceed the threshold among the remainder of the histogram bins.
6) If no histogram values exceed the threshold, then in step 60, compute the cumulative histogram c(n) wherein c(n)=h(n)+h(n−1).
7) In step 70, compute an Input/Output curve q(n), by scaling the cumulative histogram so that c(255)=255. As a result, the IO-curve thus defines a Look-Up Table (LUT).
8) In step 80, apply the IO-curve to compute new intensity values, replacing each luma pixel p(x,y) in the frame by q(p(x,y)), i.e., a pixel value defined by the IO-curve q(n).
9) In step 90, convert the pixel values in YUV, HS, or IPT to RGB tristimulus pixel values. Display the image defined by the RGB values, or use the pixel values of the image for other image processing purposes.
CLAHE methods, such as those defined by FIG. 1, have certain shortcomings, including at least some or all of the following:
The frame based CLAHE algorithm results in very dark and very bright regions that are normally unacceptable for contrast improvement.
“Local versions” of the algorithm, i.e., executing the algorithm in small regions in a larger image results in non-uniform dark and bright areas in the output image. In order to produce an image that is perceived by an observer to be of high quality, these areas need to undergo further image processing. The occurrence of sharp intensity variations in an image, i.e. non-uniform dark and bright areas in close proximity to each other, is a consequence of the use of different IO curves for different regions, which thus lack “neighborhood knowledge,” i.e. for a given area, lacking in consideration of nearby areas that are much darker or brighter.
There are conflicting requirements and/or tradeoffs in using CLAHE algorithms. In processing images via CLAHE, a given region size must be large enough, i.e., have enough pixels to make a proper histogram; however, the region size must also be small enough to adapt to local intensity variations. These constraints put limits on the region size.
The use of a high threshold on a histogram results in a highly non-linear IO-curve and is unacceptable for processing most images.
The threshold is set to be of a uniform value, which can be disadvantageous, as will be explained subsequently herein.
In uniform regions of an image such as clouds or sky, image contrast will be stretched within a very narrow intensity area on the histogram, where it is desirable that image contrast not be modified.
Also in uniform region of an image, if there is film grain noise or coding noise present, then such noise will be significantly amplified in the output image.
The brightness level of an image is not controlled when using CLAHE. An increase in output image brightness is needed in most cases in order for an observer to perceive the output image as being of high quality.
To the best of the Applicants' knowledge, CLAHE algorithms do not accept input that defines the level of contrast improvement that is desired.
In view of the shortcomings of prior art image processing and display methods, there remains a need for improving the contrast of color images, so as to provide color images that are perceived by an observer to be of higher quality.